The stupidity of marketing surveys

The New Yorker sent me a link to an important survey being conducted by some market research firm. Since I'm generally interested in how different outfits word survey questions, I figured I'd give them 10 minutes of my time and respond. I quit by the 10th item.

The survey asked me how familiar I am with a range of department stores, collecting an ordinal measurement

1. never heard of
2. heard of, but have not shopped there
3. have shopped there, but not in the last 6 months
4. shopped there in the last six months.

Among the stores on the list, other than Target(tm), I've not shopped at any of the department stores. [I'm probably not exactly the demographic that they think I am... I'm pathologically averse to spending money.] Yet, the survey gave me a screen with a bunch of check boxes asking me to check off all that apply: "When I think of Dillards, I think of..." None of the responses were appropriate; I don't even know what Dillards is, other than a store. In so far as I have a thought about the place, it's probably over-priced crap that no one needs. That, of course, is not an option. So I skipped the item. The survey responds with bold red letters... you must respond to this item. WTH?

I'd like to formally notify all those retail magnets that read this blog, don't believe anything you get from your market research surveys. If they're deployed as this survey was, the error term around any estimate they derive will be large. A much better way to measure this stuff, is give the respondent an out where they truly have no opinion. A null response is better than an invented response.

A Moment of Irony on a cold winter day

I sat at a table in the corner of the Philadelphia Center City Sheraton on Saturday working on my presentation for the Eastern Sociological Society. While working there Mitch Duneier sat down at the table next to me. He played with his blackberry for a few minutes, got up and left. Five minutes later, Eric Klinenberg sat down at the same table to eat a sandwich. What's ironic about this?

Well, in 2004 Duneier published a rather abrasive critique of Klinenberg's book Heat Wave. The substance of Duneier's critique is that Klinenberg over interprets his data and had he obtained more data (following a Symbolic Interactionst approach to theoretical sampling) he would avoid perpetuating an ethnic myth. Klinenberg responded by suggesting that Duneier's hatchet job was retaliation for a critical assessment of his own work by Loic Wacquant.

Anyway, these sorts of squabbles rise now and again with little fanfare. However, Duneier didn't let it drop. He went to Chicago (the site of Klinenberg's fieldwork) and retraced his steps. He then published a research note in the American Sociological Review summarizing his findings, that contradict Klinenberg's core argument. [We shall leave aside for the moment how Duneier was able to get a piece of work this weak published in the ASR]. Klinenberg respsonded again with a rejoinder that quite thoroughly dismantle's Duneier's argument and makes his use of data and informants look quite naive.

The tone of this latest exchange communicates pure hostility. I've read the corpus of exchanges in this squabble and though I share Duneier's commitment to Symbolic Interactionism and what Wacquant dismisses as "the modern fairytale of Grounded Theory." I think Klinenberg is in the right here.

But for all that hostility, the two can still share the same seat in the corner of the Philadelphia Sheraton... even if they do so a few minutes apart.